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Project  Objectives 

•  Identify  neural  system  features  relevant  to 

—  image  enhancement  &  object  identification  in  turbid  conditions 

—  communications  for  reconfigurable  networks  across  scales 

•  Implement  these  features  for 

—  sonar  image  processing 

—  systems  of  model  autonomous  vehicles  (AVs) 

•  Develop  and  deliver 

—  algorithms  for  mine  detection,  classification  and  identification 
(algorithm  fusion) 

—  demonstrations  of  simulated  AV  network  dynamics  based  on 
neural  system  rules  and  properties 
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Network  of  Networks 
(NoN)  Simulations 

432  Networks 

•  Platform  for  computing  at  multiple 
scales  simultaneously 

•  High  capacity  connectivity 

•  Dynamically  reconfigurable  networks 

•  Distributed,  collaborative  planning  and 
data  integration 

•  Adaptable  to  changing  environment  - 
sensory  processing,  decision  making, 
action  and  control 

•  Autonomous  operations  Sutton  JP,  Anderson  JA.  System  and 

method  for  high  speed  computing  and 
feature  recognition  capturing  aspects  of 
neocortical  computation.  U.S.  patent 
5,842,190. 1998  Nov  24. 
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Naval  Mine  Threat  and  FNC  for  MCM 

•  Threats  depend  upon  environment 

•  AVs  have  sensors,  communication,  energy,  intelligence 
and  mobility 

•  Enabling  capabilities  include  rapid  and  automated  mine 
detection,  classification  and  identification 

•  Use  of  sensor  and  algorithm  fusion 


Image  Reconstruction  Under  Turbid 
Environments 

•  Given  ultrasound  data  of  model  mine  blurred  non-uniformly 
(undistorted  images  and  dimensions  of  object  not  known) 

•  Data  consisted  of  200  contiguous  slices,  550  x  200  pixels  each 

•  Intermediate  levels  of  clustering  identified  by  a  variance 
measure  (A,  values) 


•  Gradient  decent  using  weights  which  depended  upon  the 
context  (underwater  environment)  and  A.  values 


•  3-D  reconstruction  by  course 
graining  over  16  slices 
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Initial  (blurred)  image 


X  map  revealing  space  variant 
distortion  with  uniform  profile 


Enhanced  image 


3D  overlay  with  compression 


Feature  extraction 


Guan,  Anderson,  Sutton.  IEEE  Trans  AW,  1997 


Sutton  JP,  Guan  L,  inventors.  System  and  method  for  image  regularization  in  inhomogeneous 
environments  using  clustering  in  neural  networks.  U.S.  patent  5,978,505. 1999  Nov  2. 


Pseudocolor  X  Map 
5x5  window 

X  ranges:  below  40, 40-70, 
70-100  and  above  100 


Sutton,  Sha,  Perry,  Guan.  Proc  Inter  Soc  Optical  Eng ,  1999 
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{□srB  Naval  Sensor  Data  Database 

Side  Scan  Sonar  3  Images 

•  22  400x400  pixel  images  analyzed  containing  62  targets 

•  Scaled  variance  (SV)  transformation  without  enhancement 

•  Data  compression  using  binning  of  SV  ranges  (X  map) 

•  Mine  detection  based  on  regions  of  8-20  pixels  of  uniform  A, 


•  Image 

•  Scaled  Variance  (SV)  Representation 


-  enhanced  contrast  by  transforming,  using  3x3  window: 
scaled  mean  intensity  is (/, ;)  =  X  £  W + a,  j +b) Ip  - — l(i,  j) 

a=-\  i>=-l  P 


SV  V,0'J)  =  X  + 

fl=-i  b=- 1 

rn.fi]1 
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Bin  SV  ranges  (Consolidation),  equi-  /  non-equipartitioning 
X  maps 


X=  2  Bins 

ll  2 


SV,  3x3  window 


X=  8  Bins  X=  30  Bins 

|2|3|4|5l6l7l.8  ll  1 1  1  1  I  1  1  [  H3( 


SV,  3x3  window  SV,  3x3  window 


“gh8CM “  Sha  and  Sutton>  Inf°  Sci' In  press 

Invention  Disclosure  filed  -  Automated 

Detection  Classification  /  Identification  segmentation /classification 


eNCog  Algorithms  +  GUI 
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Naval  Sensor  Data  Database 

Side  Scan  Sonar  3  Images 


mM/sea 


Results 


*  Without  training  on  sample  side  scan  targets  (identification), 
classification  of  mine-like  objects  using  3x3  window,  (3=16  in  SV 
equation,  full  SV  range,  12  non-equipartitioned  bins: 

-  TP=59,  FP=24,  FN=3  over  22  images 

-  Sensitivity  =  TP  /  (TP  +  FN)  =  95% 

-  PPV  =  TP  /  (TP  +  FP)  =  71% 

-  Typical  FP  rate  is  0.5  - 1.0  suspect  target  per  image 

*  Complementary  to  other  mine  hunting  algorithms  -  useful  for 
algorithm  fusion  CSS  Dahlgren  Division  (NSWC  visit  1  Mar  2001) 
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AVSYS  Architecture 

Collection  of  N  AVs,  Xl5  X2, ...,  XN,  where  each  AV  functions  as  an 
attractor  neural  network 

S(t)  encodes  data  characterizing  source  emitters,  targets,  other  AVs, ... 


The  degree  of  match  or  overlap  between  S(t)  and  a  template  can  be 
expressed  by  an  overlap  function  My(t),  where  i  indexes  the  template 
and  j  indexes  the  neural  network  (AV) 


For  a  network  of  AVs, 

(Mu{t)  . 

construct  an  overlap  matrix, 

M(t)  = 

*  * 

where  each  column  is  associated 
with  a  single  AV 

. 
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M(t)  is  a  function  of  time  due  to  changes  in 
the  signal  S(t) 

the  position  ij(t)  of  Xj,  which  influences  the  Xj’s  classification 
of  components  of  S(t) 

the  templates,  due  to  learning  or  external  inputs 
(e.g.,  from  a  command  center) 

Mode  0  Dynamics 

Baseline  scenario  of  winner  take  all  dynamics 

A  Vs  act  independently  and  adjust  their  position  to  increase  their 
overlap  with  the  source  component  that  most  overlaps  with  a  template 

Communication  among  the  AVs  only  occurs  when  an  AV  reaches 
max{Mjj(t)}  >  0 


Mode  1  Dynamics 

Implements  weakly  interacting  dynamics  among  the  AVs 

Each  Xj  determines  its  overlaps  M^t)  and  transmits  those  values 
exceeding  a  noise  threshold,  e,  to  all  the  other  AVs 

Classification  occurs  when  Mjj(t)  >  0  or 


max{M;j(t)}  values  are  determined 
across  AVs  (index  j)  rather  than  across  templates  (index  i) 

Allows  for  conflict  resolution 

Transient  specialization  of  AV  roles _ 
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Mode  2  Dynamics 

Implements  weakly  interacting  dynamics  among  the  A  Vs 

Similar  to  Mode  1  except  that  classification  occurs  when  M,j(t)  >  0  or 

X  Xc*(M,(r)>£f  >  e  . 

k  7=1 

More  aggressive  than  Mode  1  -  added  value  of  all  A  Vs  with  M^t)  >  e 
Transient  specialization  of  AV  roles 


AVSYS_UUV  Algorithm  and  GUI 


Completion  Time  (Cycles) 
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Completion  Time  as  a  Function  of  Information  Transfer 
and  Classification  Threshold  for  Mode  1 


10  targets 
10  AVs 


Summary  of  Rules  Mediating 
Autonomous  Network  Reconfiguration 

•  AVs  act  as  attractor  neural  networks  with  adaptive  learning 

•  Overlap  functions  elegantly  capture  complex  dynamic 
relationships 

—  between  individual  AVs  and  the  environment  (i.e.,  neural 
network  learning) 

-  among  AVs  in  the  system  (i.e.,  network  of  networks;  scalability) 

•  Overlaps  encode  ambiguity  in  the  system  -  this  is  a  critical 
feature  (and  not  a  bug)  -  it  is  the  source  of  reconfigurability 

•  Information  at  the  network  level  based  on  thresholding  in  a 
noisy  environment  (fluctuations,  incomplete  data  but  also 
inherent  ambiguity  among  overlap  functions) 
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Implications  for  Communications 

•  Amount  of  information  that  exceeds  threshold,  and  requires 
transfer  among  AVs,  is  small 

-  e  >  0.8  corresponds  to  11  (1.1%)  of  overlaps  - 10  element  vector 
1.4xl03  (1.3x10 1  %)  20 

7.7x10s  (7.2xl0'2  %)  30 

•  Contextual  disambiguation  is  key  to  how  AVSYS  works 

•  When  e  -*  1  and  9  — » 1  (i.e.,  Mode  0),  OR  when  £  — »  0  (i.e.  full 
communication  of  overlap  information  among  AYs  in  Mode  1), 
system  performance  deteriorates  (not  solved  by  unlimited 
bandwidth) 

•  Timing  of  information  transfer  is  critical  for  coordinating 
behavior  of  network  at  intermediate  scales 


Summary  of  Achievements 

04/99  -  present,  3  yr  award 

•  Established  working  team  at  6,1  with  active  6.2  connections 

•  Leveraged  ONR  support  with  NASA  /  NSBRI,  MGH  imaging 
to  achieve  objectives  on  sonar  and  AV  simulation  projects 

•  Deliverables: 

-  eNCog  software,  demo  and  documentation  for  scaled  variance 
processing  and  mine  classification  (Invention  disclosure;  Completed 
AASERT  supported  MIT  Ph.D.  thesis;  Applications  for  MCM 
algorithm  fusion) 

—  AVSYS  software,  demo  and  documentation  for  network  of  UUV 
search  (Invention  disclosure  with  UAV  software,  including 
communication  specs) 

•  Uncovered  rules  at  multiple  scales  (e.g.,  X  binning,  overlap 
functions)  within  NoN  that  have  relevance  to  reconftgurable 
networks  and  autonomy  (FNCs) 
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Future  Plans 

•  Lake  Travis  site  visit  3  May  2001  to  better  interface 
broadband  sensor  data  with  algorithms 

•  Ongoing  contact  with  CSS  as  data  source  and  evaluator  of 
algorithms 

•  Enhance  link  between  two  projects  via  UUY  and  other  AV 
systems  (intelligent  autonomy) 

•  Utilize  reconfigurable  NoN  systems  beyond  current  project 

-  ONR  /  Draper  tactical  platform  development  for  autonomous  systems 

—  NASA  /  NSBRI  smart  med  systems  and  technology  development 
(multi-scale  incl  nano-X) 

—  Automated  image  segmentation  /  classification  for  diagnostic  / 
therapeutic  radiology  and  oncology 

—  Industrial  licensing  (NewcoGen  LLC) 
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